Turning Point Prediction of Oscillating Time Series using Local Dynamic Regression 
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In the prediction of oscillating time series, the interest is in the turning points of successive 
oscillations rather than the samples themselves. For this purpose a scheme has been proposed; the 
state space reconstruction is limited to the turning points and the local (nearest neighbor) model is 
modified in order to predict the turning point magnitudes and times. This approach is extended here 
using a local dynamic regression model on both turning point magnitudes and times. Simulations 
on oscillating nonlinear systems show that the proposed approach gives better predictions of turning 
points than the standard local model applied to all the samples of the oscillating time series. 



I. INTRODUCTION 

Oscillating time series are common in applications and 
are characterized by series of patterns of an upward trend 
followed by a downward trend. When oscillating time se- 
ries do not exhibit apparent periodicity, such as those 
generated by chaotic systems, the problem of prediction 
lies basically on the time and magnitude of the peaks 
and troughs, as the results of three time series compe- 
titions showed [U, i, il. Interestingly, the winning pre- 
diction schemes in the two first competitions were based 
on a local prediction model (with rather involved modi- 
fications of the standard nearest neighbor prediction ap- 
proach) . Local models stem from dynamical systems and 
chaos theory, are computationally efhcient, and perform 
as well as more complicated black-box models, such as 
neural networks, in the prediction of irregular time series 
For multi-step ahead prediction typically a higher 
embedding dimension M is required. For a fixed time 
delay r, the reconstructed points span a time window of 
length — {M — l)r. This should be large enough to 
account for the mean orbital period of the underlying tra- 
jectory, i.e. should cover the period of an oscillation 
or a pattern of oscillations Q . 

Turning point prediction is of great practical interest in 
many applications, such as finance [1]. A recently devel- 
oped approach attempts to model oscillating time series 
from low-dimensional systems with the so-called peak-to- 
peak dynamics [3] • This approach relies on simple one or 
two dimensional maps for the peaks. In |8j|, it was shown 
that the prediction of turning points with local models 
is improved using state space reconstruction on the time 
series of turning points at a lower embedding dimension 
m. Here, we extend the state space reconstruction to 
include also the time series of the times of the turning 
points. This is the setting of local dynamic regression, 
where a local model on two time series (for magnitudes 
and times of turning points) is build in order to predict 



the magnitudes and times of turning points. 



II. STATE SPACE RECONSTRUCTION OF 
TURNING POINTS 



Suppose an oscillating time series of length N, 
{x{t)}f^i, is observed at a sampling time Ts. A sam- 
ple Ui = x{ti) is a. turning point of {x{t)}^^i at time step 
ti if it is the minimum or maximum of all samples in 
[ti — p,ti + p], for a scale parameter p. Scanning all sam- 
ples of {x{t)}^^^, the time series {yj}"=i ^^'^ {^j}"=i of 
magnitudes and times of the alternating turning points, 
respectively, are derived. Instead of the time of the turn- 
ing points we derive the duration of the upward and 
downward trends from the first differences Zi = ti — ti^i, 
giving the time series {zi}^^2- Thus two successive sam- 
ples of {yi}f=2 together with the synchronous samples of 
{^i}?=2 regard an oscillation of {x{t)}^^i, as shown in 
Fig. [H The bivariate time series {yi,Zi}"=2 compresses 
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FIG. 1: Time series of turning point magnitudes and trend 
durations derived from an oscillating time series. 



the information in {x{t)}^^l with some loss of informa- 
tion depending on the pattern of the samples between 
the turning points. At the limit where the upward and 
downward trends are linear there is no loss of informa- 
tion as any sample x{ti — k) between two turning points 
x[ti-i) and x{ti), where fc e {0, 1, . . . , <i — U-i}, can be 
expressed in terms of the magnitude and time of the two 
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turning points as 

xiti - fc) = x(ti) - k — — ^Vi - k _ . 

The state space reconstruction of {?/i}"=2 '^^'^ ^e 
considered as a specific state space reconstruction of 
{^{'t)}tLi at time points {tijf^i for delays depending at 
each ti. For an embedding dimension m, this reads 

for i = m,...,n Q. The advantage of this recon- 
struction is that it reduces the embedding dimension M 
of the standard state space reconstruction of the type 
x(t) = [x{t),x{t — Ti), . . . ,x{t — TM-i)Y to m. UsuaUy, in 
prediction tasks the delays Tj are small (and commonly 
a fixed delay t is used) suggesting a rather large M in 
order the time window to cover the mean oscillation 
period. 

We extend the state space reconstruction in ^ to ac- 
count for the duration of trends. The analysis on the 
bivariate time series {yi,Zi}2^2 requires that both time 
series are standardized (subtracting the mean and divid- 
ing with the standard deviation for each time series). The 
state space reconstruction on the standardized {yi, Zi}"^2 
reads 

= [yi,y^-i, . . . ,yt^my+i, z^, Zi^i, . . . , Zi^„i,+i]' ■ (2) 

We allow for different embedding dimensions ruy and 
niz for the magnitudes of turning points and duration 
of trends, respectively. 

III. DYNAMIC REGRESSION PREDICTION OF 
TURNING POINTS 

The prediction of y^+T f^nd z^+t for a lead time T can 
be posed independently and this constitutes a problem 
of dynamic regression (also termed distributed lag mod- 
eling) In this setting we apply local average models 
(LAM) and local linear models (LLM) Q. The predic- 
tion of yi+T with LAM is given by the average of the T 
step ahead mappings of the K nearest neighboring points 
to w., {wi(i), . . . ,w,(;^)} 

1 ^ 

Assuming a linear autoregressive model restricted to the 
neighboring points to w^, the LLM prediction of yi+T is 

yi+T^yi{K)+T + a'{wi-Wi(^K)), (4) 

where Wj(x) is the center point of the K neighboring 
points to Wi and a is estimated from the minimization of 
the error function 

K 

{yiU)+T - m(K)+T - a'(w,(j) - ^i(K))f ■ (5) 



We consider also regularization of the ordinary least 
square solution of ^ making use of principal compo- 
nent regression (PGR) and projection on the q first com- 
ponents [rO*!. Note that z^+t is predicted in the same 
way, but in line with dynamic regression setting the suit- 
able embedding dimensions my and may be differ- 
ent for the models (LAM or LLM) for yi+T and Zi+T- 
This approach differs from the approach in Q in that 
the neighboring points are formed not only based on the 
turning point magnitudes but also on the duration of 
trends. Both LAM and LLM models are simple exten- 
sions of the respective local models used for univariate 
time series. Note that the direct scheme is used here, but 
the iterative prediction scheme can be applied in a sim- 
ilar way (in [8| it was found that the iterative scheme of 
LAM on {x{ty\^^i gave worse results). In the following, 
we compare the prediction of turning points (magnitude 
and time) using LAM or LLM models estimated on all 
the samples of the oscillating time series {x{t)}^^i (de- 
noted osc-LAM and osc-LLM) and on the bivariate time 
series of turning point magnitudes and trend durations 
{2/ij2i}"=2 (denoted tur-LAM and tur-LLM). 



IV. TURNING POINT PREDICTION ON 
SIMULATED SYSTEMS 

Before presenting the results of the predictions on se- 
lected simulated systems we make some general obser- 
vations regarding the implementation of the prediction 
schemes. For a fixed number of oscillations, N is in- 
versely proportional to Tg, so that a better time resolu- 
tion of the measurements implies a larger oscillating time 
series {x{ty\^^i, whereas the length of the turning point 
time series {yi,Zi}^^2 is unaffected (being 2(n — 1)). A 
small Ts is actually welcome in the analysis based on turn- 
ing points because it allows for more accurate detection 
of the turning points and especially the trend durations. 
For example, for a time series with an average oscillation 
period of 10 samples the range of Zi is limited to integers 
from 1 to less than 10, and this range is insufficient to de- 
fine neighborhoods (in the projected reconstructed state 
space of dimension m^). Thus a smaller Ts would render 
the information of {zi}"=2 more useful in the setting of 
dynamic regression. 

The parameter p that defines the local window for the 
detection of turning points depends on r^. and should 
not be too large, so that turning points of short lasted 
oscillations can be detected, and not too small, so that 
glitches of noisy oscillations are not assigned to turning 
points. For the latter case, a small p can still be used if 
the time series is filtered, and then the turning points are 
detected on the smoothed time series to give the turning 
point times, whereas the turning point magnitudes are 
taken from the original time series. In the simulations 
we use p = 3 and filter only noisy data with an order 
depending on the system and noise amplitude. 

When predicting turning points with osc-LAM or osc- 
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LLM on {x{t)}^-^^^ the current point is not a turning 
point x{ti) but the sample x^.+p at the time the turning 
point x{ti) can first be detected. Thus using a large p 
or Ts favors the prediction on {x{t)}^-^ because then the 
current point is wch in the next trend of the oscihation. 
The prediction schemes are ihustrated in Fig. [2l Note 
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FIG. 2: Turning point prediction: the real time series seg- 
ment (grey lines and circles), the sample and turning point 
predictions with osc-LAM (dark dots and crosses), and the 
turning point prediction with tur-LAM (dark asterisks). The 
current time of the turning point is set to and the delays of 
the standard embedding on the samples are shown with open 
circles. 

that the predicted turning points with osc-LAM are de- 
tected among the multi-step sample predictions in the 
same way as the turning points are detected in the oscil- 
lating time series. 

We applied the LAM and LLM schemes on multi- 
ple realizations of known systems, such as the first and 
third variable of the Rossler system [llj, the first and 
fourth variable of the Rossler hyper-chaos system (a 
segment of this is shown in Fig. [2]), and the Mackey- 
Glass delay differential equation for different delays A = 
17,30,100 [l^. The prediction measure is the normal- 
ized root mean square error (NRMSE) of the turning 
point prediction at the last quarter of each time series. 
In Fig. [21 the average NRMSE (with the standard devia- 
tion forming the error bars) is shown for 1000 realizations 
of the fourth variable of the Rossler hyper-chaos system 
using the osc-LAM and tur-LAM models as well as the 
osc-LLM and tur-LLM models. The parameters of state 
space reconstruction for both {x{t)}^i and {yi,Zi}f^2 
were chosen so that covers up to three mean oscilla- 
tion periods. For the latter, different combinations of to^ 
and TTiz were considered and in Fig. [3] the tur-LAM and 
tur-LLM are shown for niz = 0, 1, 2 (m^ — denotes that 
the model is built only on the turning point magnitudes). 
In this example, there is little improvement of turning 
point prediction using the trend durations. Using either 
LAM or LLM, the prediction of turning points based on 
{Vi, Zi}f^2 is superior. For osc-LLM prediction of turn- 
ing point magnitudes, NRMSE is larger than one (the 
mean prediction) and has a large variance (not shown 
in Fig. [3}:). The linear mapping diverges for multi-step 
ahead predictions, and we conjecture that this is because 
temporally close points are selected in the set of the K 
neighboring points. The large variance of NRMSE is ob- 



FIG. 3: (a) The average NRMSE (with error bars for the 
standard deviation) of the prediction of next turning point 
magnitude of the fourth variable of the Rossler hyperchaos 
system (r^ = 0.1, N = 2") using osc-LAM and tur-LAM (for 
niz = 0, 1, 2 as given in the legend). The Tm in the abscissa is 
defined as = {M ~ 1)10 for osc-LAM and = (m — 1)33 
for tur-LAM, as the mean oscillation period is estimated to be 
66. (b) As in (a) but for the prediction of trend duration, (c) 
and (d) are as (a) and (b), respectively, but using the LLM 
models instead with PGR regularization parameter q = 3. 



served with all LLM models for m = 3 (equal to q) and 
this needs further investigation. 

The best predictions of LAM for turning point mag- 
nitudes and trend durations for both {x{t)}^^-^ and 
{Ui, Zi}f^2 given in Table HI The best turning point 





Turning point magnitude 
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M NRMSE 
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rUz 


NRMSE 
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0.604 
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0.505 
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0.621 
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0.518 
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10 


9 


0.662 
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0.558 
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1 


10 


0.837 


3 


1 


0.679 


2 


5 


8 


0.748 


3 


1 


0.642 


2 


10 


3 


0.732 


3 





0.665 




Trend duration 


1 


1 


10 


0.669 


2 


1 


0.368 


1 


5 


3 


0.549 


2 





0.366 


1 


10 


3 


0.526 


2 





0.414 


2 


1 


10 


1.016 


3 


1 


0.817 


2 


5 


10 


0.989 


2 





0.772 


2 


10 


9 


1.018 


2 





0.782 



TABLE I: For the system in Fig. [3] and for each combination 
of T = 1, 2 and K = 1,5, 10, the M of best prediction with 
osc-LAM and ruy and of best prediction with tur-LAM 
together with the respective NRMSE are given, where M = 
3, . . . , 10 (t = 10) and m = 2, . . . , 6. 
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predictions (magnitude and time) are derived with tur- 
LAM at small embedding dimensions (up to 3 for niy and 
or 1 for niz)- Closer investigation showed that for some 
prediction tasks osc-LAM predicted better than tur- 
LAM, whereas in other cases it formed a turning point 
far from the true turning point, so overall the NRMSE 
was worse. This difference in osc-LAM and tur-LAM per- 
sists for different A'' (we tested for log2 N = 12, 13) and 
at a lesser extent also for the addition of observational 
noise (we used 5% and 10% of noise amplitude). More- 
over, the inclusion of the last trend duration {rriz = 1) 
improved the prediction of turning point magnitudes and 
only marginally the prediction of trend durations. The 
same qualitative results were obtained from the same 
simulations on the other systems. For the highly com- 
plex Mackey- Glass system with A = 100 (it has a fractal 
dimension of about 7) the best results of tur-LAM were 
obtained for high embedding dimensions of both turning 
point magnitudes and trend durations, indicating that 
for this system the trend duration is important to pre- 
dict the next peaks or troughs. 



V. CONCLUSION 

We showed that the local prediction of turning points 
(magnitude and time) can be improved if the nearest 
neighbor model of average or linear mapping is build 
on reconstructed points from the bivariate time series 
of turning point magnitudes and trend duration. 
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